Computing Optic Flow

Opt i c f l o w i s t he apparent mo t ion o f image f e a t u r e s in the imaging p l a n e . The compu ta t i on o f o p t i c f l o w i s a f i r s t s tep i n d e t e r m i n i n g the p a r t i c u l a r mo t ions o f l i g h t i n g , o b j e c t s , and v iewers in an image sequence. New approaches are presented f o r the computa t ion o f o p t i c f l o w from dynamic image d a t a . 0 . I n t r o d u c t i o n The e s s e n t i a l i n g r e d i e n t s o f the phys ics o f v i s u a l p e r c e p t i o n are l i g h t sources , o b j e c t s to be v i e w e d , and the v iewer (eye and camera) . E l a b o r a t i n g on t h i s system we can r e l a t e b r i g h t n e s s in an image to v a r i o u s imaging f a c t o r s i n c l u d i n g i n c i d e n t i l l u m i n a t i o n , o b j e c t r a n g e , s u r f a c e r e f l e c t a n c e and o r i e n t a t i o n , camera f o c a l l e n g t h , e t c . In a dynamic imaging s i t u a t i o n imaging f a c t o r s va r y over t i m e . Th is i s u s u a l l y the r e s u l t o f the mot ion o f o b j e c t s , v i e w e r s , o r l i g h t sou rces . A l l e l s e be ing e q u a l , these mo t ions induce the co r respond ing mot ions o f t h e i r p r o j e c t i o n s on the imaging p l a n e . Opt ic f l o w i s the apparent mo t ion of image f e a t u r e s in the imaging p lane and can be rep resen ted as a v e c t o r f i e l d in t h a t p l a n e . The compu ta t i on o f t h i s f l o w f i e l d i s a f i r s t s tep i n d e t e r m i n i n g the p a r t i c u l a r mo t ions o f l i g h t i n g , o b j e c t s , and v iewers in an image sequence. Cur ren t methods f o r comput ing o p t i c f l o w f a l l i n t o two c a t e g o r i e s . The f i r s t , f e a t u r e m a t c h i n g , i n v o l v e s d e t e c t i n g f e a t u r e s in image frames and match ing them in success ive frames [ 1 , 6 , 8 ] . The second method a t temp ts to " sense" o p t i c f l o w d i r e c t l y f rom, the dynamic image da ta [ 2 , 1 , 7 ] . Th is method has a l s o been seen to have two s tages [ 2 , 1 , 7 , 8 ] . I n the f i r s t s tage the component o f o p t i c f l o w p e r p e n d i c u l a r to an edge ( o r p a r a l l e l to the image g r a d i e n t ) i s measured l o c a l l y . These v e c t o r s w i l l b e c a l l e d o p t i c f l o w c u e s . I n the second s tage the mutual c o n s t r a i n t s o f o p t i c f l o w cues are used to compute the t r u e o p t i c f l o w . Th is paper p resen ts new approaches to bo th of these s t a g e s . 1. Computing Op t i c Flow Cues 1.1 Image Grad ien t Methods Fennema and Thompson, and Horn and Schunck d e r i v e a computa t ion f o r o p t i c f l o w cues based on the assumpt ion o f cons tan t b r i g h t n e s s ( i n t ime) o f an o b j e c t ' s image [ 2 , 1 ] . I f the dynamic image i s rep resen ted as a sca la r f i e l d I ( x , y , t ) , t hen they g i v e the magni tude o f the component o f o p t i c f l o w i n the d i r e c t i o n o f the (2 -D) b r i g h t n e s s g r a d i e n t ( I x , I y ) a s where ( I X , I y I t ) i s the g r a d i e n t o f I ( x , y , t ) . T h i s computa t ion a l s o assumes t h a t the b r i g h t n e s s i s smoothly v a r y i n g in X and Y. Th is requ i remen t is v i o l a t e d when the l o c a l d e t e c t i o n window c o n t a i n s any one o f the f o l l o w i n g : 1) An o c c l u s i o n boundary between o b j e c t s w i t h d i f f e r e n t v e l o c i t i e s 2) A shadow boundary w i t h a v e l o c i t y d i f f e r e n t from t h a t o f the shadowed o b j e c t 3) A specu la r r e f l e c t i o n ( h i g h l i g h t ) on an o b j e c t moving r e l a t i v e t o t h a t o b j e c t A l l o f these cases i n v o l v e moving b o u n d a r i e s . I t i s proposed here t h a t the d e t e c t i o n o f the mot ion cues at these boundar ies must be computed w i t h respec t to some edge o p e r a t o r . 3D~edge o p e r a t o r s are i n t e r e s t i n g i n t h a t they p r o v i d e bo th the edge and the mot ion cue i n f o r m a t i o n . 1.2 A n a l y s i s of Dynamic Images in 3D XYT space D i s c o n t i n u i t i e s (edges) in the image r e s u l t from d i s c o n t i n u i t i e s i n the imaging f a c t o r s , e g . o c c l u s i o n o f o b j e c t s , shadowing ( o c c l u s i o n o f l i g h t i n g ) , s u r f a c e m a r k i n g s , and i n t e r n a l edges o f o b j e c t s (see Table 1 ) . Table 1 Imaging f a c t o r su r face range s u r f a c e o r i e n t a t i o n i n c i d e n t l i g h t i n g s u r f a c e r e f l e c t a n c e Discountinuity o c c l u s i o n boundary i n t e r n a l o b j e c t edge shadow boundary su r face mark ings